In many applications involving video footage, it is useful to be able to determine if a sequence of video frames (i.e. the individual images which, when viewed sequentially, make up a sequence of moving images) of the video footage can be considered to be static. For example, it is a common requirement for video advertisements intended for television or other similar media to have a frozen sequence of frames (i.e. a sequence of substantially identical frames) at the end. It is also useful to be able to check video footage for unexpected sequences of frozen frames, which is a common feature of video footage which has been incorrectly converted from analogue to digital formats.
It is important that any method for determining whether a sequence of video frames can be considered to be static is robust in the presence of noise in the video footage or signal of interest. Such noise typically arises during analogue capture and transmission processes, and in the process of digitally compressing the video footage. It is also important that any such method is able to detect changes in only a small part of the video frames making up the video footage.
One approach to determining if a sequence of video frames can be considered to be static is to calculate an RMS difference value between pairs of adjacent frames (referred to as the reference frame and the subsequent frame) in the sequence. In this method, the difference between a value of each pixel of the reference frame and a value of each corresponding pixel of the subsequent frame is calculated. These differences are squared, and the sum of the resulting squares is calculated and averaged, by dividing the sum of the squares by the number of pixels in a single frame. The square root of the result of this averaging operation is then calculated and compared to a threshold to assess whether the differences between the frames are within acceptable limits, indicating that the subsequent frame is substantially the same as the reference frame. This test is repeated for all of the frames in the sequence, and if all of the frames are found to be substantially the same as their adjacent frames, the frame sequence can be considered to be static.
A disadvantage of this approach is that if the threshold is made large enough to exclude the effects of noise from the assessment of whether the subsequent frame is substantially the same as the reference frame, movement in a small part of the frames (i.e. differences in pixel values between a small part of the subsequent frame and a corresponding part of the reference frame) may not be detected, thus giving rise to false positive assessments that the frame sequence can be considered to be static.
An alternative approach is to calculate the difference between corresponding pixels of the reference and subsequent video frames and compare each of these individual differences to a threshold. A test is then made to calculate the proportion of the individual differences, for the entire video frame, which exceed the threshold, and if this proportion is below a predetermined maximum for all of the frames in the sequence, the frame sequence can be considered to be static.
A difficulty with this approach is that it is not very sensitive to differences between frames which are caused by noise, and is also not particularly sensitive to movement in a small part of a frame.
Additionally, both of the approaches described above are computationally intensive, as the calculation of the difference between corresponding pixels of the first and subsequent frames must be carried out for all of the pixels making up a frame.